The present invention relates generally to an interface between an operator and a machine. It relates more specifically to the field of such interfaces which present a signal to a human operator in contact with the interface. The invention relates most specifically to an interface that presents or exhibits a force signal to an operator, such as a human, or receives a force signal from such an operator. Because a force signal is by definition bi-directional, it can be said that the interface and the user xe2x80x9cexchangexe2x80x9d a force signal, or xe2x80x9csharexe2x80x9d it with each other.
Machines are ubiquitous in modem life, and every machine must be controlled, either directly or indirectly by a human operator. The interface through which the operator controls the machine and receives information from the machine should be as easy to use as possible, in light of the functionality the machine provides. Examples of such machines include slave robotic machines that operate in an environment different from that in which the operator exists. Other machines include machine tools for shaping materials, vehicles and powered machinery. Computers are also machines, which manipulate data representing, among other things, text (word processors); numbers (spread sheets); records (data bases); geometrical constructs (drawing and painting programs), etc.
The user may control and otherwise interact with such machines through various devices, such as a lever, joystick, mouse (having buttons and a tracking mechanism), exoskeleton, keyboard, touch screen, digitized pad or tablet, head mouse, etc. Typically, the user manipulates a xe2x80x9cmasterxe2x80x9d input device in the user""s local environment and the xe2x80x9cslavexe2x80x9d robot, typically in a different environment, moves in accordance to the user""s manipulations. The configuration of the master device may or may not conform to some degree to the conformation of the slave device.
For a rigid body, such as a rod-like appendage of a machine, the number of freedoms necessary to unambiguously specify its relation to a reference frame is typically considered to be six. Conceptually, three freedoms specify the location of a point on the rigid body, relative to the reference frame, and three additional freedoms specify the orientation of the rigid body relative to the same, or an equivalent reference frame.
Components on master devices typically are provided with numerous degrees of freedom of motion to permit varied motions by the user. The number of such degrees can be from one to six or more. These numerous freedoms are facilitated by, numerous joints and actuators. Thus, a master arm may have a hand portion, with several fingers, each with several joints. The hand may be joined through a wrist joint to a forearm section, joined through an elbow joint to an upper arm section, joined through a shoulder joint to a trunk. Considering the joint of a finger most distant from the trunk, it""s state relative to a reference frame can be specified by six freedoms, three for its position and three for its orientation.
However, the entire arm assembly may have many more than these six freedoms, due to the numerous joints and their various flexibilities. There may be several conformations of the other elements of the arm that place the terminal finger digit in the same state. Many, or all of the actuators that drive the arm may contribute to establishing the state of a single freedom, such as the location along one axis. Thus, the entire arm itself has many freedoms, more than six. However, only six freedoms of motion are required to specify the state of any rigid body portion of the arm.
Certain of such master and slave machine systems, known as force reflecting systems, provide actuators such that motions of the master component through the various degrees of freedom are affected or constrained to some extent. Typically, the motions are affected based on conditions in the environment of the slave robotic machine, such as forces that the slave encounters. Thus, the user, grasping or otherwise contacting the master machine, experiences constraints on the freedoms of motion that relate in some way to the slave environment, and thus, receives a force feedback signal. A teleoperator is such a device.
In certain instances, it is desirable for the user to feel the forces as if the user were contacting the slave environment directly, rather than remotely through the master to slave connection, including intervening stages. A system that accomplishes this is sometimes referred to as a xe2x80x9cforce reflectingxe2x80x9d system. Such a force reflecting interface is also referred to as a xe2x80x9chapticxe2x80x9d interface because it relates to the human system of touch. Typical design considerations for such an interface include the fidelity of the position and force or torque feedback, simplicity of structure, minimization of backlash, independence of freedoms of motion, work space conformation, stiffness, responsiveness, sensitivity, minimization of the physical bulkiness of apparatus and the bandwidth of its response. By bandwidth, it is meant, the range of combinations of speed of response and force applied.
In addition to controlling traditional, physical machines, it is known for human operators to control xe2x80x9cvirtualxe2x80x9d machines and environments, which are not physical, but rather are xe2x80x9cembodiedxe2x80x9d or reside in a computer model.
Simple examples abound in connection with common computer tasks. For instance, using a computer drawing or painting program, a user controls a group of virtual geometric objects that can be moved relative to one another, created, destroyed, altered, stretched, etc. Another example is the now familiar xe2x80x9cdesktopxe2x80x9d metaphor for showing a directory of computer files, and for the user to provide instructions with respect to the manipulation (copying, deleting, opening, modifying, etc.) of those files. Within a word-processing program, the user manipulates virtual controls to scroll through different parts of the text of a document, to delete (xe2x80x9ccutxe2x80x9d) certain sections and to add (xe2x80x9cpastexe2x80x9d) them elsewhere. There are many more examples. Basically, such examples include anything where a user affects representations of data elements, as represented by the computer interface.
More complicated examples include those in which a more realistic environment is created, such as by using more sophisticated visual renditions of objects and settings, and projection devices such as helmets and special eyeglasses.
A user may interact with the virtual environment by means of various physical input devices, such as have been mentioned above. Sound may also be a part of the interface.
The virtual, or artificial environments may also recreate or simulate real environments, and can be used for the practice of skills, such as medical surgery, geological excavation, dangerous cargo manipulation, etc.
The various interactive systems may expand the abilities of humans, by increasing physical strength, improving manual dexterity, augmenting the senses, and by projecting human users into remote and abstract environments, either real or artificial. The remote environments can also be of a scale much larger or much smaller than typical human scales.
Force reflecting systems can be differentiated from other known simulations, such as graphical flight simulators, and remote controls, by the provision of force feedback. To enhance the user""s perception of physical interaction with the slave environment, more than visual and auditory cues are required. Touch, is the only one of the five human senses that provides a two way interface with the environment. Using touch, a human can affect the environment while at the same time, perceiving the effect of the contact with the environment. Such direct feedback facilitates the user""s perception of presence or influence in the slave environment. In effect, with touch, a force signal is exchanged or shared between the user and the machine, just as equal and opposite forces are shared by two people holding hands.
The purpose of the force reflecting master is to give a user the sense that the user is touching an object that is not actually in the local environment. The object, referred to below as a xe2x80x9cnon-localxe2x80x9d object, can be a real object being manipulated by a physical slave machine, or it can be a representation in an environment that exists only as a computer data model.
For an ideal haptic interface, the user would not realize that he was touching an interface separate from the environment to be manipulated. Specifically, a user would not be able to distinguish between touching a real object and touching a virtual object with the device. Further, the device would not encumber the user. The ideal interface would exert no external force on the user when the user is moving freely in space.
Hard surfaces, such as walls, should feel as stiff with the device as they do in real life, even when contacted at high velocity. Comers of solid objects should feel crisp. Compliant surfaces should feel springy.
Some known attempts at constructing force reflecting interfaces have used an xe2x80x9cexoskeleton.xe2x80x9d An exoskeleton is worn by the user and can often exert forces at several locations along the arms and/or fingers. See generally, B. A. Marcus, B. An, and B. Eberman, xe2x80x9cEXOS Research on Master Controllers for Robotic Devices,xe2x80x9d FIFTH ANNUAL WORKSHOP ON SPACE OPERATIONS APPLICATIONS AND RESEARCH (SOAR ""91) pp. 238-245, July 1991. There are many constraints in the design of an exoskeleton device, because the structure must attach to several locations on the human body and the exoskeleton joints must effectively be co-located with human joints. Counterbalancing such structures, and designing stiff, uncoupled transmissions for them is difficult. The structures must be counterbalanced so that the user does not perceive them as an artificial construct of the feedback system. The transmissions must be stiff so that there is a feeling of direct contact with the non-local environment.
Another type of force reflecting interface uses an externally grounded joystick. Typical of these devices are the traditional xe2x80x9chot-cellxe2x80x9d manipulator systems and force reflecting hand controller.
Thus the several objects of the invention include, to enable human interaction with a non-local environment, either physical or computer represented, with a high degree of realism. It is an object to facilitate a high fidelity position and torque or force feedback, so that the user has an accurate perception of the conditions in the non-local environment. The user interface should be transparent to the user and as unobtrusive as possible. Implicit in this object is to minimize system backlash. It is further an object to provide such an interface that permits user action over a physically appropriate size of workspace, without necessitating a bulky or overly complicated apparatus. It is also an object to provide a device that responds quickly enough to conditions in the non-local environment for a realistic simulation, and which displays appropriate stiffness and sensitivity, as well as a relatively large response bandwidth, so that relatively quick motions can be perceived and imparted by the user. It is also an object of the invention to display discontinuous events, such as impact.
In a preferred embodiment, the invention is an apparatus for physically exchanging a force with a user in an environment local to the user. The apparatus comprises a connection element for physically connecting to a user""s body member and a linkage between the connection element and ground. The linkage includes means for powering at least three independent freedoms of the connection element relative to ground and means for maintaining at least one independent freedom of the connection element relative to said ground free of power. Up to three independent freedoms of the connection element may be maintained free of power, and up to five independent freedoms may be powered, although the number of powered and free freedoms of the connection element alone does not exceed six. The linkage may also include three linked bearings, with two pairs of the three bearings being orthogonal, such as a gimbal assembly. The axes of the bearings intersect at a reference point. For instance, the connection element can be a thimble, for insertion of a user""s finger, with the intersection point being inside the user""s finger, as connected to the thimble.
The linkage may also include at least two masses that are movable relative to ground and each other and the connection element, such that the center of mass among these items remains substantially stationary relative to ground despite motion of the connection element. The masses may constitute actuators, which may be connected to a local grounded element through a single cable transmission. Other user connection elements include a rod or stylus, or thimbles sized to accept other body members, such as the head, buttocks, foot, hand, arm, leg, tongue and toe.
It is also sometimes beneficial to track the motions of the freedoms that are unpowered. The ground may be a portion of the user""s body other than that to which the connection element is connected.
In another preferred embodiment, the powered freedoms are tracked and a signal is generated based on the tracking of the freedoms. The signal is transmitted to a non-local environment. The non-local environment may be a physical environment or a virtual, computer resident environment.
In yet another preferred embodiment, the invention is an apparatus for physically exchanging a force with a user in a first environment that is local to the user. The apparatus comprises a user connection element and a linkage for connecting the element to ground. The linkage includes a pair of quarter gimbals with the connection element fixed to a rotational bearing fixed to one end of one of the quarter gimbals. The free end of the other quarter gimbal is connected to an extension of one bar of a five bar linkage. The five bar linkage is actuated by two actuators, each connected between a different one of the bars of the linkage and a support that is more proximal to ground than the actuators. The support is actuated with respect to ground by a third actuator. The three actuators combine to power three freedoms of the connection element. The gimbals combine to maintain three freedoms of the connection element free of power.
Still another embodiment of the invention is an apparatus for generating a signal at a specified point. The apparatus comprises a pair of actuators that are both connected to ground through the same cable. A linkage is also provided for kinematically connecting both of the actuators to the specified point. The actuators may both be connected between ground and the specified point through a five bar linkage.
Another embodiment of the invention is an apparatus for generating a signal representative of a force, in effect, a virtual switch. The apparatus comprises a receiver for receiving a signal representative of the location of a user reference point relative to a user reference frame and a model for storing a representation of: a non-local reference frame; the user reference frame, relative to said non-local reference frame; and the conformation of a non-local environment comprising a switch-type, spring-type element, relative to said non-local reference frame. A comparator is provided for comparing the location of the user reference point relative to the non-local environment. A force generator is provided for generating a signal representative of a force. The force signal is based on the location of the user reference point relative to the non-local environment and a set of force rules. The force rules include spring-force rules which specify a switch output force signal in response to a location signal of the user reference point indicative of a deflected conformation of the spring-type element. The switch output force signal is specified by a non-linear function. Thus, a realistic virtual switch is provided. The invention may also include an operator that makes changes to the representation of the non-local environment based on the signal representative of force and the set of force rules. For instance, the representation of the switch changes location in the non-local environment.
Another preferred embodiment of the invention is a similar apparatus for generating a signal representative of a force, where the non-local environment comprises a type of element which changes its cross sectional area in response to a force in a direction perpendicular to the plane of the area. Such an element is defined as a xe2x80x9cdiagonalxe2x80x9d type element. Such elements include a bristle brush, or a sponge. With such an embodiment, which is similar to the virtual switch embodiment, the force rules include a spring-force rule that specifies a diagonal element output force signal in response to a location signal of the user reference point indicative of a deflected conformation of the diagonal-type element. This simulates the feeling that a user has when pushing against a deforming bristle head of a brush. The operator for calculating changes to the non-local environment based on the force signal specifies a change in the representation of the cross-sectional area of a selected region of said diagonal-type element. The non-local environment may also include an indicia of the cross-sectional area of the selected region of said diagonal-type element, analogous to the mark a paint filled brush makes when pressed onto a painting substrate. The apparatus may also include means for storing this indicia over time, thus storing a painted line of widths that vary along its length, based on the force applied by a user. The non-local environment may also include a plurality of such force rules, analogous to different sizes and stiffnesses and shapes of brushes.
In another preferred embodiment of the invention, the force generator can generate forces based on the time history of the location of the user reference point relative to the non-local environment. The force rules include friction-type rules which specify a friction output force signal in response to the time history of the location signal of the user reference point indicative of a change in position over time of the user reference point.
Another preferred embodiment of the invention is similar to the previous embodiment. The non-local environment includes a representation of a drafting substrate over which the reference point moves. The force rules, rather than specifying rules for the moving reference point, specify force generating rules for a substrate. A force generator is provided for generating a signal representative of a force, based on the location of the user reference point relative to the non-local environment and a set, of force rules. The rules include drafting substrate-force rules, which specify a drafting substrate output force signal in response to a location signal of the user reference point indicative of a deflected conformation of the drafting substrate-type element. The apparatus may also include a non-local environment reaction calculator that makes changes to the representation of the conformation of the non-local environment based on the signal representative of force and the set of force rules. The drafting substrate type-element rule specifies a change in the representation of a surface shape of a selected region of the drafting substrate-type element. The surface texture of the substrate may be included in the non-local environment, and may be modeled as a rippled wall.
Another preferred embodiment of the invention is A method for physically exchanging a force between an apparatus and a user in a first, user-local environment, said method comprising the steps of providing an apparatus as described above, having a connection element for physically connecting to a body member of a user and a linkage between the connection element and ground. The linkage comprises means for powering at least three independent freedoms of the connection element relative to ground and means for maintaining at least one independent freedom of the connection element relative to ground free of power. The method also includes the steps of connecting the connection element to a body member of the user and powering the at least three independent freedoms of the connection element.
Another preferred embodiment of the invention is a method for generating a signal representative of force, such as for a virtual paint brush. The method comprises the steps of receiving a signal representative of the location of a user reference point relative to a user reference frame. Another step is storing a representation of: a non-local reference frame; the user reference frame, relative to the non-local reference frame; and the conformation of a non-local environment comprising a diagonal-type, spring-type element, relative to the non-local reference frame. The location of the user reference point is taken relative to the non-local environment. A signal is generated representative of a force, based on the location of the user reference point relative to the non-local environment and a set of force rules. The force rules include spring-force rules which specify a diagonal element output force signal in response to a location signal of the user reference point indicative of a deflected conformation of the diagonal-type element. The representation of the conformation of the non-local environment is changed based on the signal representative of force and the set of force rules. The diagonal element spring-type rule specifies a change in the representation of the cross-sectional area of a selected region of the diagonal-type element.